1. Field of the Invention
The present invention generally relates to character entry and recognition systems and, more particularly, to use of character template on an electronic writing surface to structure a dialogue between the user and a computing system during the character entry phase so as to simplify and make more reliable the character recognition phase.
2. Description of the Prior Art
Portable notebook computers utilizing handwriting interfaces are thought to offer substantial benefits to users in circumstances where keyboards may not be used. Such computers incorporate software systems for recognizing handwritten characters and converting them to a coded form which is conventionally "understood" by computers. Modern handwriting recognition systems are at best insufficient for many of these applications, hence, frequent error correction steps are necessary by the user to obtain the desired text. The larger the number of steps required to generate an accurate sentence, the less usable the system is.
Prior art systems correct errors using one or more of the following strategies: 1) rewriting of character(s) in error; 2) alternate recognized character selection; 3) using a soft keypad to generate correct character. The first strategy has been found to fail often. The second strategy is easy to use, but if the desired character is not in the list an additional strategy must be employed. The third strategy is highly accurate, but requires a large percentage of screen space to display all possible character choices.
Current recognition systems are limited as to the forms of handwriting they can recognize. Cursive writing recognition is not generally available. Recognition of cursively written characters is complicated by the fact that the characters are written as a single connected line. Separation of the characters at the inter-character ligatures is often difficult even for humans. Detecting this separation and then recognizing the separated character bas been computationally expensive for computerized recognition systems.
Prior relevant art may be divided into two categories. In one category are those which deal with entry of characters on paper followed by recognition with an electronic apparatus. These methods resemble the present invention to the extent that the phases of character creation and character recognition are clearly separated and that the user must be satisfied with the production before submitting it to recognition. However, in each of these methods the flow of information through the system is unidirectional. The user does not see any production of the computing system until the recognition is complete. Furthermore, in order to simplify what would otherwise be an extremely difficult recognition task, most of these methods directly constrain the handwriting style of the user.
U.S. Pat. No. 3,485,168 to Martinson discloses a means for accurately writing characters to be submitted for "optical character recognition". In this system, an instrument is used to "develop" the elements of a pattern placed on a work surface. The characters are thus accurately formed and registered, which simplifies the electronic part of the recognition system. However, the Martinson invention guides the user to create optically recognizable characters by the appearance of lines only in the event that the user applies the "writing instrument" to a pretreated region of the work sheet. No line appears which follows the locus of the stylus in any arbitrary area to which the stylus is applied. Thus the failure to correctly "develop" a line is indicated to the user by the appearance of incomplete line traces. There is no mechanism for determining that the line traced by the stylus is close to a predefined line segment; instead, the line segments themselves are made very wide, which limits the "template" to highly stylized line segments. Furthermore, no correction means for inadvertently marked lines is described by the Martinson patent. The claims of the Martinson patent explicitly call for a physical "work sheet" and "ink".
In U.S. Pat. Nos. 4,132,976 and 4,275,381 to Siegal, there are disclosed character recognition systems incorporating a matrix of pre-inscribed circular marks on a work sheet. These are also paper and ink based systems. Here, instead of chemically treated regions which force the markings to be in well registered locations, as in the Martinson patent, the user is expected to memorize a character set and form each character so that parts of the character fall within designated pre- inscribed circular marks. As with the Martinson patent, there is no dialogue in which the computer system and the user agree upon the form of the character to be recognized.
In the article "Computer Identification of Constrained Handprinted Characters with a High Recognition Rate" (IEEE Transactions on Systems, Man, and Cybernetics: November, 1974; p.497ff), a method of handwriting recognition is described in which "49 constrained characters" are enterable by hand printing the constrained characters on a grid printed on paper, then placing that paper in the view of a computer equipped with a TV camera input. The grid in this system is nearly identical to the template disclosed in the present invention. However, the system described in the article is an optical character recognition system. There is no dialogue between the computer and the user to agree upon the character to be recognized. Rather, the system uses a feature extraction and best match recognizer to identify the character.
In U.S. Pat. No. 3,776,454 to Jones a five by three array of light colored boxes is provided at regular intervals on a dark background. The user may enter a decimal number by blackening those boxes which do not belong to the number. Here again there is no dialogue between a computer system and the user.
Japanese Patent Nos. 62-20086, 62-20087, and 62-20089 deal with a method for producing written characters which can be read easily by an optical character reader. In 62-20086 and 62-20087 the user is provided with an array of lines which may be traced to create numerals. In 62-20089 a pattern of rectangles is provided which, when used with a described character set, support accurate reproduction of that character set. The patent does not address any technology for recognizing the user production but only offers a tool for regularizing the production so that recognition is easier. As with the earlier discussed patents, there is no computer-user dialogue in which the computer and the user agree upon the form of the character to be recognized.
In U.S. Pat. No. 3,733,468 to Eberly, there is disclosed a means for manually producing alphanumeric characters readable by an optical scanner. The line segments of a template are embossed as grooves on paper, and the grooves then guide the stylus point forcing regular characters to be formed. This system can not be recreated on a handwriting capture tablet without the addition of mechanical templates or robotic devices. As with much of the prior art already discussed, this invention depends upon constraints which conform user writing strokes to predefined alternatives. The Eberly patent achieves this constraint with the aid of a mechanical device. While this technique allows for much simpler character recognition devices, it does not take advantage of the natural handwriting skills of users.
A second category of prior art teaches character entry on an electronic device.
An article entitled "Handprinted Data Entry with a Touch-Sensitive Numeric Keypad" (Proceedings of the Human Factors Society 33rd Annual Meeting, 1989, p.305ff by Georg Geiser) describes a methodology for constructing a set of 30 "line elements" on a three by three array of buttons by a certain "stroking" motion. These line elements are used to "write" characters via the array of buttons. The recognition system "decodes" a series of button presses. The stroke path is necessarily constrained to a small keypad. There is no difference between the user entered "stroke" and the displayed segments. The methodology described cannot easily be expanded to support cursive handwriting entry or logic or flow chart capture.
U.S. Pat. No. 4,005,400 to Engdahl discloses a stylus-operated two-by-two switch array for generating numbers on a small pad, such as might be found on a watch calculator. The patent describes how different numbers can be generated by different sequences of "strokes" on the array. There is no dialogue between the computer system and the user, and character recognition decoding is based upon the sequence in which the "strokes" are made.
In U.S. Pat. No. 4,199,751 to Piguet, a variety of small keyboards are described for character entry. The keyboard buttons are so arranged that they correspond to the segments of a single character display. The user creates characters by pushing buttons. No use is made of the natural handwriting style of the user.
U.S. Pat. No. 3,487,371 to Frank discloses a terminal for block character entry and recognition. The user is expected to print on paper, within boxes which may contain "reference markings which constrict the writing of the characters in the box." While this system does present a user interface for handwriting input and a mechanism for displaying to the user each entered character as it is recognized, it has no dialogue phase. "Contour following" and "feature extraction" techniques are used to recognize the entered characters.